Apparatus and method for visualization of region of interest

ABSTRACT

There is provided an apparatus for visualizing a region of interest (ROI) in a Computer Aided Diagnosis (CAD) system. The apparatus includes: an image receiver configured to receive images; an ROI acquirer configured to acquire the ROI from a current image; and an ROI visualizer configured to, in response to acquisition of the ROI from the current image, output visualization information for visualizing the ROI acquired from the current image based on a change between the ROI acquired from the current image and an ROI acquired from a previous image.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit under 35 USC 119(a) of Korean PatentApplication No. 10-2014-0100677, filed on Aug. 5, 2014, in the KoreanIntellectual Property Office, the entire disclosure of which isincorporated herein by reference for all purposes.

BACKGROUND

1. Field

The following description relates to an apparatus and method ofvisualizing a region of interest (ROI), and more particularly to atechnology using a Computer Aided Diagnosis (CAD) system that is allowedto visualize an ROI.

2. Description of Related Art

In a case of ultrasound examination using an ultrasonic device, such asa fetal ultrasound, an abdominal ultrasound, and a breast ultrasound,both a doctor and a patient look and review a result of the examinationon a screen. In general, ultrasonic image diagnosis techniques aredesigned mainly for measuring a size of a region of interest (ROI) in astill image, visualizing and storing a marking, or displaying andstoring an annotation, rather than providing an explanation forpatients. Thus, the patients rely on the doctor's explanation to checkand understand the result of the ultrasonic examination. However, due tothe patient's unfamiliarity with ultrasonic images, it is difficult forthe patient to understand which part of an image a doctor is describingand thus the patient may fail to identify the character or significanceof the part of the image the doctor is describing. In addition, when apatient requests a more detailed explanation, a doctor needs to takeextra actions, for example, the doctor may need to move a probe untilthe patient can notice a change on an image displayed on the screen, andprovide a verbal explanation, or point out an image on the screen usinga finger or a pointer.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In one general aspect, there is provided an apparatus of visualizing aregion of interest (ROI), the apparatus including: an image receiverconfigured to receive images; an ROI acquirer configured to acquire theROI from a current image; and an ROI visualizer configured to, inresponse to the acquisition of the ROI from the current image, outputvisualization information for visualizing the ROI acquired from thecurrent image based on a change between the ROI acquired from thecurrent image and an ROI acquired from a previous image.

The ROI acquirer may comprise an ROI detector configured toautomatically detect one or more ROIs by applying a detection algorithmto the current image.

The ROI acquirer may comprise an input receiver configured to acquireone or more ROIs from the current image based on an input.

The ROI visualizer may comprise: an interest item identifier configuredto, in response to the acquisition of the ROI from the current image,identify whether an interest item exists in the ROI acquired from thecurrent image; and an interest item determiner configured to determinewhether the interest item in the current image is identical to aninterest item detected from the previous image, when the interest itemexists in the ROI acquired from the current image.

The interest item identifier may be further configured to identifywhether the interest item exists in the ROI acquired from the currentimage, by extracting, from the ROI acquired from the current image,features comprising one or more of shape, brightness, texture, andcorrelation with surrounding areas, and classifying an image pattern ofthe ROI based on the extracted features.

The interest item determiner may be further configured to, in responseto the images received being continuous, determine whether the interestitem in the current image is identical to the interest item detectedfrom the previous image based on a difference in the ROI acquired fromthe current image and the ROI acquired from the previous image.

The interest item determiner may be further configured to, in responseto the images received being not continuous, determine whether theinterest item in the current image is identical to the interest itemdetected from the previous image by matching the interest item in thecurrent image with a three-dimensional (3D) object generated for theinterest item in the previous image.

The ROI visualizer may comprise: an ROI output configured to outputvisualization information of the ROI acquired from the current imagebased on a result obtained by the interest item identifier and adetermination made by the interest item determiner.

The ROI output may be further configured to adjust the visualizationinformation based on either or both of a number and a size of interestitems, when the interest item exists in the ROI acquired from thecurrent image.

The ROI output may be further configured to, in response to adetermination that the interest item in the ROI acquired from theprevious image does not exist in the ROI acquired from the currentimage, remove visualization information of the ROI acquired from theprevious image from a screen displaying the ROI.

The ROI output may be further configured to, in response to adetermination that the interest item in the current image is identicalto the interest item detected in the previous image, outputvisualization information of the ROI acquired from the previous image asvisualization information for visualizing the ROI acquired from thecurrent image.

The ROI output may be further configured to, in response to adetermination that the interest item in the current image is notidentical to the interest item detected from the previous image, outputnew visualization information that is distinguishable from visualizationinformation of the ROI acquired from the previous image.

The visualization information may be generated by combining firstinformation, which comprises a square, a circle, a free curve, a crossand an arrow, with second information, which comprises color, a linetype, and line thickness.

In another general aspect, there is provided a method of visualizing aregion of interest (ROI), the method comprising: receiving images;acquiring the ROI from a current image; and, in response to theacquisition of the ROI from the current image, outputting visualizationinformation for visualizing the ROI acquired from the current imagebased on a change between the ROI acquired from the current image and anROI acquired from a previous image.

The acquiring of the ROI may include detecting one or more ROIs based onan input and automatically detecting the one or more ROIs by applying adetection algorithm to the current image.

The method may further include: in response to the acquisition of theROI from the current image, identifying whether an interest item exitsin the ROI acquired from the current image; and determining whether theinterest item in the ROI acquired from the current image is identical toan interest item detected from the previous image, when the interestitem exists in the ROI acquired from the current image.

The determining of whether the interest item in the ROI acquired fromthe current image is identical to an interest item from the previousimage may include, in a case where the images received are continuous,determining whether the interest item in the current image is identicalto the interest item detected from the previous image based on adifference between the ROI acquired from the current image and the ROIacquired from the previous image.

The determining of whether the interest item in the ROI acquired fromthe current image is identical to an interest item from the previousimage may include, in a case where the images received are notcontinuous, determining whether the interest item in the current imageis identical to the interest item detected from the previous image bymatching the interest item in the current image with a three-dimensional(3D) object generated for the interest item in the previous image.

The outputting of the visualization information may include adjustingthe visualization information based on either or both of a number and asize of interest items and outputting the adjusted visualizationinformation, when the interest item exists in the ROI acquired from thecurrent image.

The outputting of the visualization information may include removingvisualization information of the ROI acquired from the previous imagefrom a screen displaying the ROI, when the interest item in the ROIacquired from the previous image does not exist in the ROI acquired fromthe current image.

The outputting of the visualization information may include, in responseto a determination that the interest item in the current image isidentical to the interest item detected from the previous image,re-outputting visualization information of the ROI acquired from theprevious image as visualization information for visualizing the ROIacquired from the current image.

The outputting of the visualization information may include, in responseto a determination that the interest item in the current image is notidentical to the interest item detected from the previous image,outputting new visualization information that is distinguishable fromvisualization information of the ROI acquired from the previous image.

Other features and aspects may be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating an apparatus of visualizing aregion of interest (ROI) according to an embodiment.

FIG. 2 is a block diagram illustrating an ROI acquirer shown in FIG. 1.

FIG. 3 is a block diagram illustrating an ROI visualizer shown in FIG.1.

FIGS. 4A to 4E are diagrams illustrating examples of visualization of anROI.

FIG. 5 is a flowchart illustrating a method of visualizing an ROIaccording to an embodiment.

FIG. 6 is a flowchart illustrating an ROI acquiring operation shown inFIG. 5.

FIG. 7 is a flowchart illustrating an ROI visualizing operation shown inFIG. 5.

Throughout the drawings and the detailed description, unless otherwisedescribed, the same drawing reference numerals will be understood torefer to the same elements, features, and structures. The relative sizeand depiction of these elements may be exaggerated for clarity,illustration, and convenience.

DETAILED DESCRIPTION

The following description is provided to assist the reader in gaining acomprehensive understanding of the methods, apparatuses, and/or systemsdescribed herein. Accordingly, various changes, modifications, andequivalents of the methods, apparatuses, and/or systems described hereinwill be suggested to those of ordinary skill in the art. Also,descriptions of well-known functions and constructions may be omittedfor increased clarity and conciseness.

Throughout the drawings and the detailed description, the same referencenumerals refer to the same elements. The drawings may not be to scale,and the relative size, proportions, and depiction of elements in thedrawings may be exaggerated for clarity, illustration, and convenience.

The features described herein may be embodied in different forms, andare not to be construed as being limited to the examples describedherein. Rather, the examples described herein have been provided so thatthis disclosure will be thorough and complete, and will convey the fullscope of the disclosure to one of ordinary skill in the art.

Hereinafter, an apparatus and method of visualizing a region of interest(ROI) are described with reference to the following drawings.

FIG. 1 is a block diagram illustrating an apparatus of visualizing aregion of interest (ROI) according to an embodiment. An apparatus 100 ofvisualizing an ROI, shown in FIG. 1, may be a component of aComputer-Aided Diagnosis (CAD) system that receives images in sequenceand performs a diagnosis on the received image. According to anembodiment, the apparatus 100 may be part of a CAD system that analyzesand diagnoses ultrasonic images received in real time from a probe, asdescribed below. However, aspects of the present disclosure are notlimited thereto.

Referring to FIG. 1, the apparatus 100 includes an image receiver 110,an ROI acquirer 120, an ROI visualizer 130, and an image output 140.Although FIG. 1 illustrates the image receiver 110, the ROI acquirer120, the ROI visualizer 130 and the image output 140 as included in theapparatus 100, these components may be embodied as independent hardware.Therefore, the apparatus 100 illustrated in FIG. 1 is not limitedthereto and thus may include more or less components.

The image receiver 110 receives images in sequence. The image receiver110 may receive images of an examination area of a patient from an imageacquiring device in real time. There may be a plurality of imageacquiring devices, and an image acquiring device may be an ultrasoundexamination device that converts a signal of a patient, such as abiological signal, measured by a probe, into an electronic signal andconverts the electronic signal into an image. The output electronicsignal may change in time, and the image acquiring device may visualizethe electronic signal in real time and transmit the visualization resultto the image receiver. In addition, according to an embodiment, imagesreceived in sequence may be real-time input images in units of frames.

The image output 140 outputs an image received by the image receiver110. According to an embodiment, if images are received in sequence fromthe image receiver 110, the image output 140 outputs the current imageon a screen, while making a previous image disappear on the screen.According to another embodiment, the image output 140 may output thecurrent image in a predetermined area of the screen, while outputting ona different area of the screen an image that is designated by a userfrom among previous images as an image that needs to be maintained. Thedesignation may be in response to a result obtained by the interest itemidentifier 131 and a determination made by the interest item determiner132. In this case, when a diagnostic result is generated throughanalysis of a received image, the diagnostic result may be output in apredetermined area of the screen. The diagnostic result may overlay thecurrent image that is already being output on the screen.

If the image receiver 110 receives images in sequence, the ROI acquirer120 may acquire an ROI from the currently received image according to apredetermined standard, and calculate a location and size of the ROI.The ROI may indicate an area where an interest item exists or issupposed to exist in the image. The interest item may include, forexample, a lesion, an embryo, or a fetus finger/toe, but is not limitedthereto. In this case, the interest item may be preset according to adiagnostic purpose.

The ROI visualizer 130 visualizes an ROI input by a user or acquiredthrough automatic detection and outputs the ROI on a screen. Once a userdesignates an ROI in an image output on the screen, the ROI visualizer130 may visualize the designated ROI by outputting preset visualizationinformation on the screen. In this case, the visualization informationmay be previously generated by combining first information, whichincludes, for example, a square, a circle, a free curve, a cross, and anarrow, with second information, which includes, for example, a color, aline type, and line thickness. In addition, according to a type of eachinterest item and whether each interest item is first detected, thevisualization information may be previously combined by a user's input,and then stored.

For example, if a user touches a screen with a finger or a touch pen,the ROI visualizer 130 may visualize an ROI by displaying a cross at thecenter of the touched point or by displaying a distinguishing marker,such as, for example, a square and/or a circle, on the boundary of anarea at a predetermined distance away from the center of the touchedpoint. In addition, if a user draws a boundary of an ROI, the ROIvisualizer 130 may output a distinguishing marker on the edge of thetouched point by using, for example, various colors, line types, andline thickness, all of which may be preset by a user.

FIG. 2 is a block diagram illustrating an ROI acquirer shown in FIG. 1.

Referring to FIG. 2, the ROI acquirer 120 may include a user inputreceiver 121 and an ROI detector 122. Although FIG. 2, illustrates theuser input receiver 121 and the ROI detector 122 included in the ROIacquirer 120, the user input receiver 121 and the ROI detector 122 maybe embodied as independent hardware. Therefore, the ROI acquirer 120illustrated in FIG. 2 is not limited thereto and thus the ROI acquirer120 may include more or less components.

The user input receiver 121 receives various types of information thatis input by a user using various devices. The various devices mayinclude a probe, a mouse, a touch pen, and a finger. The user inputreceiver 121 may provide a user interface and may receive various userinputs through the user interface.

According to an embodiment, the user input receiver 121 may receive andset various reference information from a user, which may be necessaryfor visualization of an ROI. For example, the user input receiver 121may receive an ROI acquisition condition and a detection algorithm forautomatic detection. Herein, the ROI acquisition condition may include‘automatic acquisition,’ which indicates automatically detecting an ROIusing a detection algorithm, and ‘manual acquisition,’ which indicatesacquiring an ROI based on a user's input. The detection algorithm mayinclude Deformable Parts Model (DPM). In addition, in the case ofautomatic detection, the reference information may include a thresholdof possibility to be an interest item, which may be necessary foracquiring an ROI, and the maximum number of ROIs to be output on thescreen.

In addition, the user input receiver 121 may receive and setvisualization information from a user in order to visualize the acquiredROI. The visualization information received from the user may includethe first information, which indicates a type of distinguishing marker,such as a square, a circle, a free curve and a cross, and the secondinformation, which indicates details of the distinguishing marker, suchas a color, a line type, and line thickness. By doing so, the user isable to set and change a distinguishing marker and details thereof.

According to another embodiment, in the case where an ROI acquisitionstandard is ‘manual acquisition’, the user input receiver 121 mayacquire an ROI by receiving a user's input. If a user designates one ormore areas where any interest item is likely to exist in the currentlydisplayed image suspected, the user input receiver 121 may acquire anROI by calculating a location and size of the area. In this case, theuser may designate an ROI of appropriate size according to a size of theinterest item by checking the image currently displayed on the screen.That is, the user may designate an ROI by drawing a square, a circle, ora free curve at the boundary of the interest item on the screen. Inaddition, if the user designates a center of the interest item, the userinput receiver 121 may acquire an area of preset-sized area as an ROI byautomatically calculating location information of the center of theinterest item.

In the case where an ROI acquisition standard is ‘automaticacquisition’, the ROI detector 122 may automatically detect an ROI byapplying a preset detection algorithm to an image each time a new imageis received. In this case, the ROI detector 122 detects one or more ROIsfrom the current image. If a plurality of ROIs are detected, the ROIdetector 122 may calculate a possibility of each ROI to include anyinterest item, and then acquire an area as an ROI if a possibilitythereof is greater than a preset threshold (i.e. 50%). In this case, themaximum number of ROIs (e.g., five ROIs) to be output on a screen may bepreset according to a resolution of the screen. If the number of ROIseach having a possibility greater than a preset threshold level (e.g.,50%) exceeds the maximum number, the ROI detector 122 may acquire ROIsin descending order of possibilities of ROIs to include any interestitem.

Based on a user's input, the user input receiver 121 may acquire an ROIfrom among the ROIs detected by the ROI detector 122 as an ROI to beconstantly traced from among subsequent images. In this case, ifvisualization information of ROIs automatically detected by the ROIdetector is output on a screen, the user may check each ROI and selectany area including a desired interest item as an ROI to be traced.

FIG. 3 is a block diagram illustrating an ROI visualizer shown in FIG.1.

Referring to FIG. 3, the ROI visualizer 130 may include an interest itemidentifier 131, an interest item determiner 132, and an ROI output 133.Although FIG. 3 illustrates the interest item identifier 131, theinterest item determiner 132 and the ROI output 133 included in the ROIvisualizer 130, these components may also be embodied as independenthardware. Therefore, the ROI visualizer 130 illustrated in FIG. 3 is notlimited thereto and thus the ROI visualizer 130 may include more or lesscomponents.

The interest item identifier 131 identifies whether any interest itemexists in an ROI acquired by the ROI acquirer 120. The interest itemidentifier 131 may identify whether any interest item exists in theacquired ROI, by extracting features from the acquired ROI and thenclassifying an image pattern of the ROI using the extracted features. Inthis case, the extracted features may include the ROI's shape,brightness, texture, and correlation with surrounding areas.

According to an embodiment, the interest item identifier 131 may utilizea feature extractor and a classifier. Each of the feature extractor andthe classifier may be a software program or hardware equipment, andthere may be one or more feature extractors/classifiers. In addition,the feature extractor and the classifier may be included in the ROIidentifier 131. Alternatively, the feature extractor and the classifiermay be included in an additional hardware device or in a CAD device towhich the apparatus 100 is utilized.

The feature extractor converts a feature into a numeric value andoutputs the numeric value. The output feature consists of a featurevector having numerous values and may be changed according to a receivedimage. To expedite the identifying process, the feature extractor mayextract features only from an area of the current image, which has adifference from a previous image. Alternatively, the feature extractormay use the same features as those extracted from a previous image.

The classifier may be a module that is generated by extracting featuresin each ROI from an image database in advance and performing machinelearning on the extracted features. The classifier classifies an imagepattern of an ROI by using an image feature vector extracted from theROI. If an interest item is a lesion, the image pattern may bebenign/malignant. According to a type of interest item and a diagnosticpurpose, there may be various image patterns. The image pattern may beimage patterns.

According to another embodiment, the interest item identifier 131 mayidentify the existence of an interest item by performing a similaritysearch to search an image similar to the current image.

If the interest item identifier 131 identifies that any interest itemexists in an ROI acquired from the current image, the interest itemdeterminer 132 determines that the identified interest item in thecurrent image includes an interest item detected from a previous imagein order to trace the interest item. That is, if an interest itemdetected from the currently received image is identical to an interestitem has been constantly detected from previous images, the interestitem determiner 132 continuously traces the interest item and the ROIoutput 133. The continuously tracing consists of visualizing the ROI inorder to notify that the interest item is being traced.

According to an embodiment, based on a degree in change of a receivedimage, the interest item determiner 132 may determine whether theinterest item detected from the current image is identical to aninterest item detected from a previous image. For example, based on adifference in intensity of ROIs, a difference in histograms, asimilarity in histograms, or a difference in a location/angleinformation of ROIs between the current image and the previous image,the interest item determiner 132 may determine whether the interestitems are the same.

According to another embodiment, the interest item determiner 132 maydetermine whether the interest item detected from the current image isidentical to an interest item detected from previous images by matchinga three-dimensional (3D) object relating to an ROI acquired from aprevious image with a cross-section of the current image.

If a received image is one of images continuously received in real time,the interest item determiner 132 may make a determination based on adegree in change of images. Alternatively, if a received image is adiscontinuous image, the interest item determiner 132 may make adetermination by matching a 3D object.

In response to a determination that an interest item detected from anROI in the current image is a new interest item that has not beendetected from any previous image, the interest item determiner 132 mayprocess information on the interest item. For example, the interest itemdeterminer 132 may store location and angle information of the newlydetected interest item in a storage device, such as a memory and a disc.In addition, each time a new interest item is detected from an imagereceived in sequence, the interest item determiner 132 may generate a 3Dobject by performing 3D modeling on the new interest item.

At a time when the current image is output on a screen, the ROI output133 outputs visualization information for visualizing an ROI detectedfrom the current image. That is, if one or more ROIs are detected by auser's input or detected automatically, the ROI output 133 outputsvarious kinds of visualization information in surroundings of eachacquired ROI to notify a user or a patient of the detection of the ROI.If a plurality of ROIs are detected, the ROI output 133 may distinguishthe ROIs from each other with different numeric values or colors. Forexample, the order of numeric values to be output or darkness of a linemay be set differently according to a size of ROIs and a possibility ofeach ROI to include an interest item. For example, in descending orderof size of ROIs, 1, 2, 3, . . . , and N may be attached in sequence,red, orange, yellow and blue may be displayed in sequence, or linethickness may gradually become thin. Likewise, the same display methodmay be applied to the ROIs in descending order of possibilities of theROIs to include an interest item.

In addition, if a user manually designates an ROI on a screen in a casewhere the ROI acquisition condition is a manual acquisition, the ROIoutput 133 may output, on the screen, the designated ROI in a presetform, such as a square, a circle, and a free curve.

If a plurality of ROIs are detected automatically in a case where theROI acquisition condition is automatic detection, the ROI output 133outputs visualization information of each detected ROI. Then, if a userselects only some of the detected ROIs, the ROI output 133 may removevisualization information of unselected ROIs to notify that the selectedROIs are to be analyzed and traced. Alternatively, the ROI output 133may change visualization information of the selected ROIsdistinguishably from those of unselected ROIs.

In addition, based on a result obtained by the interest item identifier131 and a determination made by the interest item determiner 132, theROI output 133 may output visualization information for visualizing onlyan ROI detected from the current image.

For example, if the interest item identifier 131 identifies that aninterest item exists in an ROI acquired from the current image, the ROIoutput 133 may adjust visualization information to be output based onthe number and size of interest items included in the ROI and may outputthe adjusted visualization information. That is, as described above, ina case where there are a plurality of interest items, it is possible tooutput visualization information of an ROI including one interest itemdistinguishably from that of a different ROI including another interestitem. In addition, it is possible to dynamically adjust a size ofvisualization information of an ROI in proportion to a size of aninterest item included therein.

In another example, if the interest item determiner 132 determines thatan interest item detected from a previous image exists in the currentimage, visualization information of the previously detected interestitem, which was once output, may be re-output to notify a patient and adoctor that the corresponding interest item was previously detected. Newvisualization information of each newly detected interest item may beoutput distinguishably from that of the previously detected interestitem to notify the patient and the doctor that a new interest item isdetected.

According to the embodiments, visualization information is removed ormaintained depending on whether an ROI and/or interest item exists in anultrasonic image and on whether an interest item was previouslydetected. In addition, the visualization information of a newly detectedinterest item is output with a new color, shape, form, or the like,which are distinguishable from those of a previously detected interestitem. As a result, a user and a doctor are able to recognize eachother's interest items.

FIGS. 4A to 4E are diagrams illustrating examples of visualization of anROI. Herein, (t) denotes the current point in time. From the currentpoint in time, (t−b) denotes a point in time prior to time b, (t−a)denotes a point in time prior to time a, and (t+b) denotes a point intime after time b. In addition, a point in time may be a unit of frames.

FIG. 4A is an example of ROI visualization in a case where an interestitem is a lesion and an ROI is acquired based on the user's input. InFIG. 4A, when the user designates an ROI in surroundings of an interestitem in an image received at (t−a) (left image), preset visualizationinformation of a square 41 is accordingly output. At this point, theoutput square 41 may be output with a particular color, for example,yellow, predetermined by the user from among various colors. Then, ifimages are received in sequence and the interest item disappears in theimage received at (t) (the second and third images from the left),visualization information of the ROI is removed from the screen, asshown in the third image from the left. At this point, it is possible tomake the square 41 seem to disappear, not instantly, but gradually bychanging the shape of the square 41 (the second image from the left), asshown in the second image from the left, gradually blurring the outputcolor, or changing a type of the line of the square 41. Then, images arereceived continuously, and, if the interest item designated by the userat (t−a) is re-detected from an image at (t+b) (the image at the right),the square 41, which was output at (t−a), is output again as square 42at (t+b).

FIG. 4B is another example of ROI visualization in a case where aninterest item is a lesion and an ROI is acquired through automaticdetection. In FIG. 4B, when an image at (t) (the left image) isreceived, an ROI is detected from the image through automatic detection,and when three ROIs are acquired as a result, squares 51, 52, and 53 forthe respective three ROIs are output on the screen (corresponding to thesecond image from the left). At this point, size of the squares 51, 52,and 53 may be determined in accordance with a size of the interest item.In addition, the squares 51, 52, and 53 may be output with differenttypes of colors and lines or with the same type of color and line. Then,if a user selects any one of the squares, for example, the square 53, itis possible to notify that the square 53 is the ROI finally selected bythe user, by changing a type of color or line of the square 53(corresponding to the third image from the left) while removing thesquares 51 and 52 for the ROIs not selected by the user (correspondingto the image at the right).

FIG. 4C is another example of ROI visualization in a case where aninterest item is a lesion and an ROI is acquired based on the user'sinput. In FIG. 4C, if an image at (t−b) is received and output on ascreen (the left image) and a user touches a point suspected to be anROI, a dotted-line square 61 centered at the touched point is output(corresponding to the second image from the left). Then, in order tonotify that the touched point is acquired as an ROI, the dotted-linesquare 61 may be changed to have a different type of line and/or colorand maintained for a while, and then may disappear (corresponding to thethird image from the left). Then, images are received continuously anddiagnosis is performed (the right image at the bottom). If an interestitem first appears in the ROI designated in the image at (t), a square62 centered at the interest item is output. At this point, the size ofthe square 62 may be dynamically adjusted in proportion to a size of theinterest item.

FIG. 4D is another example of ROI visualization in a case where aninterest item is a fetus finger or toe and a user designates an ROI in areceived image. Likewise, an abdominal ultrasonic image is received andoutput at (t−a) (corresponding to the left image on the top). If a userdesignates an area suspected to include a fetus, a dotted-line square 71is output (corresponding to the right image at the top). Then, if a nextimage is received, the square 71 disappears and diagnosis on the nextimage is performed. If a fetal finger and toe is found in the currentimage received at (t), arrows 72 and 73 of different colors are output(corresponding to the right image at the bottom). Then, if the interestitem is continuously detected in subsequent images, the arrows 72 and 73output with respect to the interest item at (t+b) continues to be output(corresponding to the left image on the bottom). At this point, locationand size of an arrow in an image received in real time may be changedaccording to location and size of the interest item. In addition, if theinterest item disappears or appears in the image, the arrows 72 and 73may thereby disappear or appear.

FIG. 4E is another example of ROI visualization in which an ultrasonicimage is acquired by scanning the same area of examination with a probein a different scanning direction. The upper part of FIG. 4E illustratesa procedure in which a user performs diagnosis by scanning an area ofexamination in a top-to-bottom direction. Specifically, the user sets anROI ({circumflex over (1)}), scans an area of examination in atop-to-bottom direction ({circumflex over (2)}), traces the ROI({circumflex over (3)}), stores information on location and size of adetected interest item when the trace is finished, and performs 3Dmodeling of the interest item to generate a 3D object of the interestitem ({circumflex over (4)}). That is, an interest item is constantlytraced from the point in time when the interest item was first detectedfrom a previous image, and a 3D object of the interest item may begenerated by rendering volume information of the interest item.

The lower part of FIG. 4E illustrates a procedure where a user performsdiagnosis by scanning the same area of examination in a left-to-rightdirection. A user acquires a real-time image by moving a probe from theleft side to the right side ({circumflex over (1)}), and, in response toan interest item found at a location from an arbitrary distance from thepreviously detected interest item in the upper side, matches theinterest item with the previously generated 3D object in order todetermine whether the interest item is identical to the previouslydetected interest item ({circumflex over (2)}). If a determination ismade that the interest item is identical or similar to the previouslydetected interest item, visualization information, which was once outputfor the previously detected interest item, is output again, and theinterest item may be traced ({circumflex over (3)}).

FIG. 5 is a flowchart of a method of visualizing an ROI according to anembodiment. FIG. 6 is a flowchart illustrating an ROI acquiringoperation shown in FIG. 5. FIG. 7 is a flowchart illustrating an ROIvisualizing operation shown in FIG. 5.

FIGS. 5 to 7 may be examples of a method of visualizing an ROI by theapparatus shown in the embodiment of FIG. 1. Since various embodimentshave been already described in detail, descriptions thereof arehereinafter omitted.

In operation 310, the apparatus 100 receives images in sequence. Theimages received in sequence may be ultrasonic images in units of frames,which are acquired in real time by a probe.

In operation 320, the apparatus 100 acquires an ROI from the currentlyreceived image. At this point, the ROI may be acquired automatically ormanually based on a user's input.

Referring to FIG. 6, operation 320 is described in detail. In operation321, the apparatus 100 checks a preset ROI acquisition condition. TheROI acquisition condition may be preset by a user, and may includeautomatic detection and manual detection. In the case of automaticdetection, it is possible to set a detection algorithm, a threshold forpossibility of each ROI to include an interest item, and the maximumnumber of ROIs to be output.

If it is found in operation 321 that the preset ROI acquisitioncondition is automatic detection, a detection algorithm is applied tothe currently received image to detect one or more ROIs in operation322. The one or more detected ROIs are output with various types ofvisualization information, as described below.

Then, if a user inputs selection of a desired ROI in operation 323 afterchecking visualization information of each ROI on the screen, theapparatus 100 may acquire the selected ROI as an ROI to be traced inoperation 324. At this point, if user does input selection of none orall of the detected ROIs, the apparatus 100 may acquire an automaticallydetected ROI as an ROI to be diagnosed and traced.

Alternatively, if it is found in operation 321 that the ROI acquisitioncondition is manual detection, the apparatus 100 waits to receive auser's input. If the user identifies an interest item and designates anROI in the current image in operation 323, the apparatus 100 acquiresthe designated ROI as an ROI to be diagnosed and traced in operation324. At this point, the user may designate an ROI using variousinputting methods. For example, the user may designate an ROI bytouching the center of an area suspected to include any interest item orby drawing a square, a circle, or a free curve around the suspectedarea.

Again referring to FIG. 5, if an ROI is acquired from the currentlyreceived image, the apparatus 100 visualizes the ROI in operation 330.Specifically, if the ROI is acquired based on a user's input or throughautomatic detection, the apparatus 100 may output, on the edge of theROI, visualization information such as a square, a free curve, an arrowand a cross of various colors, line types, and line thickness.

Referring to FIG. 7, operation 330 of visualizing an ROI is described inmore detail. In operation 331, whether any interest item exists in anROI of the current image is identified. In a case where it is found inoperation 322 that any interest item does not exists in an ROI of thecurrent image, whether any interest item exists in an ROI of asubsequent image is identified in operation 331. Whether any interestitem exists in an ROI may be identified by extracting features from thecurrent image and classifying an image pattern based on the extractedfeatures.

If it is found in operation 332 that an interest item exists in the ROIacquired from the current image, whether the interest item in thecurrent image is identical to an interest item detected from a previousimage is determined in operation 333. If received images are continuousimages input in real time, whether the interest item in the currentimage is identical to an interest item detected from a previous imagemay be determined based on a change between the ROI acquired from thecurrent image and an ROI acquired from the previous image.Alternatively, if the received image is a discontinuous image, i.e., animage scanned in a different direction of the same area, whether theinterest item in the current image is identical to an interest itemdetected from a previous image may be determined by matching theinterest item in the current image with a 3D object previously generatedfor the interest item in a previous image.

If it is determined in operation 333 that the interest item is notidentical to the interest item detected from the previous image,information on the interest item is processed in operation 334. Forexample, information of a newly detected interest item, such as locationand size thereof, is recorded and managed in a storage device, and 3Dmodeling of the newly detected interest item is performed to generate a3D object. In addition, the newly detected interest item or an ROIincluding the same is output in operation 335 to notify a user of thedetection.

If it is determined in operation 333 that the interest item is identicalto an interest item detected from a previous image, visualizationinformation that was output regarding the previous image is output againwith respect to an ROI in the current image in operation 336.

The methods and/or operations described above may be recorded, stored,or fixed in one or more non-transitory computer-readable storage mediathat includes program instructions to be implemented by a computer tocause a processor to execute or perform the program instructions. Thenon-transitory computer-readable storage media may also include, aloneor in combination with the program instructions, data files, datastructures, and the like. Examples of non-transitory computer-readablestorage media include magnetic media, such as hard disks, floppy disks,and magnetic tape; optical media such as CD ROM disks and DVDs;magneto-optical media, such as optical disks; and hardware devices thatare specially configured to store and perform program instructions, suchas read-only memory (ROM), random access memory (RAM), flash memory, andthe like. Examples of program instructions include machine code, such asproduced by a compiler, and files containing higher level code that maybe executed by the computer using an interpreter. The described hardwaredevices may be configured to act as one or more software modules inorder to perform the operations and methods described above, or viceversa. In addition, a non-transitory computer-readable storage mediummay be distributed among computer systems connected through a networkand computer-readable codes or program instructions may be stored andexecuted in a decentralized manner.

A number of examples have been described above. Nevertheless, it shouldbe understood that various modifications may be made. For example,suitable results may be achieved if the described techniques areperformed in a different order and/or if components in a describedsystem, architecture, device, or circuit are combined in a differentmanner and/or replaced or supplemented by other components or theirequivalents. Accordingly, other implementations are within the scope ofthe following claims.

What is claimed is:
 1. An apparatus to visualize a region of interest (ROI), comprising: at least one processor; and at least one memory storing one or more computer programs that, upon execution by the at least one processor, configure the at least one processor to: receive images in sequence via a probe, acquire at least one ROI comprising at least one pre-designated interest item from one (hereinafter, previous image) among the images, output visualization information for visualizing the at least one ROI acquired from the previous image on the previous image, detect at least one ROI from a current image received by a movement of the probe, identify, in response to detection of the at least one ROI from the current image, whether an interest item exists in the at least one ROI detected from the current image, remove, in response to a determination that the interest item does not exist in the at least one ROI detected from the current image, the visualization information of the at least one ROI acquired from the previous image, determine, in response to a determination that the interest item exists in the at least one ROI detected from the current image, whether the interest item existing in the current image is identical to the pre-designated interest item in the previous image, and output the visualization information of the at least one ROI acquired from the previous image as visualization information for visualizing the at least one ROI detected from the current image on the current image based on a result of the determination.
 2. The apparatus of claim 1, wherein the at least one processor is further configured to: automatically acquire one or more ROIs by applying a detection algorithm to the previous image.
 3. The apparatus of claim 1, wherein the at least one processor is further configured to: acquire one or more ROIs from the previous image based on a user's input.
 4. The apparatus of claim 1, wherein the at least one processor is further configured to: identify whether the interest item exists in the ROI detected from the current image, by extracting, from the at least one ROI detected from the current image, features comprising one or more of shape, brightness, texture, and correlation with surrounding areas, and classifying an image pattern of the at least one ROI based on the extracted features.
 5. The apparatus of claim 1, wherein the at least one processor is further configured to: determine, in response to the images received in sequence being continuous, whether the interest item existing in the current image is identical to the pre-designated interest item in the previous image based on a difference in the at least one ROI detected from the current image and the at least one ROI acquired from the previous image.
 6. The apparatus of claim 1, wherein the at least one processor is further configured to: determine, in response to the images received in sequence being not continuous, whether the interest item existing in the current image is identical to the pre-designated interest item in the previous image by matching the interest item existing in the current image with a three-dimensional (3D) object generated for the pre-designated interest item in the previous image.
 7. The apparatus of claim 1, wherein the at least one processor is further configured to: adjust the visualization information of the at least one ROI acquired from the previous image based on either or both a number and size of interest items, when the interest item exists in the at least one ROI detected from the current image, and output the adjusted visualization information as the visualization information for visualizing the at least one ROI detected from the current image.
 8. The apparatus of claim 1, wherein the at least one processor is further configured to: output, in response to a determination that the interest item existing in the current image is identical to the pre-designated interest item in the previous image, the visualization information of the at least one ROI acquired from the previous image as the visualization information for visualizing the at least one ROI detected from the current image on the current image.
 9. The apparatus of claim 1, wherein the at least one processor is further configured to: output, in response to a determination that the interest item existing in the current image is not identical to the pre-designated interest item in the previous image, new visualization information that is distinguishable from the visualization information of the at least one ROI acquired from the previous image as the visualization information for visualizing the at least one ROI detected from the current image on the current image.
 10. The apparatus of claim 1, wherein the visualization information is generated by combining first information, which comprises a square, a circle, a free curve, a cross and an arrow, with second information, which comprises color, a line type, and line thickness.
 11. A method of visualizing a region of interest (ROI), comprising: receiving images in sequence via a probe; acquiring at least one ROI comprising at least one pre-designated interest item from one (hereinafter, previous image) among the received images; outputting visualization information for visualizing the at least one ROI acquired from the previous image on the previous image; detecting at least one ROI from a current image received by a movement of the probe; identifying, in response to detection of the at least one ROI from the current image, whether an interest item exists in the at least one ROI detected from the current image; removing, in response to a determination that the interest item does not exist in the at least one ROI detected from the current image, the visualization information of the at least one ROI acquired from the previous image; determining, in response to a determination that the interest item exists in the at least one ROI detected from the current image, whether the interest item existing in the current image is identical to the pre-designated interest item in the previous image; and outputting the visualization information of the at least one ROI acquired from the previous image as visualization information for visualizing the at least one ROI detected from the current image on the current image based on a result of the determination.
 12. The method of claim 11, wherein the acquiring of the at least one ROI comprises one of: acquiring one or more ROIs based on a user's input; or automatically acquiring one or more ROIs by applying a detection algorithm to the previous image.
 13. The method of claim 11, wherein the determining comprises, in a case where the images received in sequence are continuous, determining whether the interest item existing in the current image is identical to the pre-designated interest item in the previous image based on a difference between the at least one ROI detected from the current image and the at least one ROI acquired from the previous image.
 14. The method of claim 11, wherein the determining comprises, in a case where the image received in sequence are not continuous, determining whether the interest item existing in the current image is identical to the pre-designated interest item in the previous image by matching the interest item existing in the current image with a three-dimensional (3D) object generated for the pre-designated interest item in the previous image.
 15. The method of claim 11, wherein the outputting of the visualization information for visualizing the at least one ROI detected from the current image comprises: adjusting, when the interest item exists in the at least one ROI detected from the current image, the visualization information of the at least one ROI acquired from the previous image based on either or both a number and size of interest items; and outputting the adjusted visualization information as the visualization information for visualizing the at least one ROI detected from the current image.
 16. The method of claim 11, wherein the outputting of the visualization information for visualizing the at least one ROI detected from the current image comprises, in response to a determination that the interest item existing in the current image is identical to the pre-designated interest item in the previous image, re-outputting the visualization information of the at least one ROI acquired from the previous image as the visualization information for visualizing the at least one ROI detected from the current image on the current image.
 17. The method of claim 11, wherein the outputting of the visualization information for visualizing the at least one ROI detected from the current image comprises, in response to determination that the interest item existing in the current image is not identical to the pre-designated interest item in the previous image, outputting new visualization information that is distinguishable from the visualization information of the at least one ROI acquired from the previous image as the visualization information for visualizing the at least one ROI detected from the current image on the current image.
 18. A computer program product comprising a non-transitory computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to: receive images in sequence, acquire at least one ROI comprising at least one pre-designated interest item from one (hereinafter, previous image) among the images, output visualization information for visualizing the at least one ROI acquired from the previous image on the previous image, detect at least one ROI from a current image received by a movement of the probe, identify, in response to detection of the at least one ROI from the current image, whether an interest item exists in the at least one ROI detected from the current image, remove, in response to a determination that the interest item does not exist in the at least one ROI detected from the current image, the visualization information of the at least one ROI acquired from the previous image, determine, in response to a determination that the interest item exists in the at least one ROI detected from the current image, whether the interest item existing in the current image is identical to the pre-designated interest item in the previous image, and output the visualization information of the at least one ROI acquired from the previous image as visualization information for visualizing the at least one ROI detected from the current image on the current image based on a result of the determination. 